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Abstract 

Motivational effects during a simulated educational game should be studied because a general concern of lecturers is 
motivating students and increasing their knowledge. Given advances in internet technology, traditional short in-class 
games are being substituted with long web-based games. To maximize the benefits of web-based simulation games, a 
game should instill intrinsic motivation in students and encourage them to adopt deep-learning strategies. We have 
conducted a quasi-experimental study of 53 students, divided into game and no-game groups, as part of an operations 
management course. During the course, the game group played a web-based simulation game lasting 7 days; the 
no-game group did not play this game at all. The findings revealed that students who played the game experienced a 
deeper level of learning. However, students who did not play the game felt more competent and exerted greater efforts. 
These results are useful for challenging the assumed benefits of educational games at motivating students. Furthermore, 
our findings open up opportunities for finding more factors that affect the motivation and behavior of individual 
students. 
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1. Introduction 

1.1 Web-Based Simulated Game in Teaching Operations Management 

In general, simulations are the “art and science of creating a representation of a process or system for the purpose of 
experimentation and evaluation” (Gogg & Mott, 1993). In educational institutions, simulations are designed in the form 
of games to combine education with leisure. Simulation games in teaching operations management adopt a 
problem-based learning (PBL) approach (Kanet, 2007; Naik, 2011). The PBL approach requires lecturers in operations 
management to introduce students to real-world business operations problems that need to be diagnosed and solved. 
Simulation games are composed of a model or “an abstraction of an economic environment” and a simulation that is 
“the series of rules for manipulation of the model” and a game or “a set of rules which govern the activity of the 
participants in relation to the simulation” (McKenney & Dill, 1966). “Web-based simulation represents the connection 
between the web and the field of simulation” (Fishwick, 1996). Internet technology is bringing real business into 
classrooms via web-based games, which stimulate the active engagement of students to diagnose problems in particular 
contexts, analyze provided information, and formulate an implementation plan. We use a web-based simulation game 
called Littlefield Laboratories developed by the Responsive Learning Technologies Company based in the United States. 
The game, designed in the form of a goal-based scenario, “allows students to systematically refine their understanding 
and intuition of system behavior through exploration and iterative experimentation” (Responsive Learning 
Technologies). 

The game aims to maximize the cash balance for Littlefield Lab, which provides blood-testing services using a four-step 
process (Fig. 1). Students assemble into teams and spend exactly 7 days, equivalent to 268 simulated days, operating the 
lab by accessing a team account on the internet. The winning team will have the largest cash balance in the end of the 
simulation term. The game is designed in a highly-integrative manner in which teams have to diagnose problems related 
to demand forecasting, capacity planning, inventory lot sizing, reorder point level determination, and sequencing. The 
students have to prepare a plan to address problems. When the game is over, the students will complete a report that 
assesses their plan. A lecturer or a tutor simply plays a facilitating role during the game. This educational game offered 
as pilot innovative part of operations management syllabus for business undergraduate students in the School of 
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Business - International University - Vietnam National University from 2012 to 2014. 



THE LABORATORY 

Figure 1. Production Process of the Littlefield Laboratories (Source: Responsive Learning Technologies) 


1.2 Study Objective 

The objective of the study is to assess the impact of the web-based simulated game practiced in operations management 
courses on students’ intrinsic motivation and their use of deep learning strategy at university level in Vietnam. This 
impact is also compared to that of no-game conventional teaching approach. The findings of the study will confirm or 
reject the benefits of this game in generating positive study behavior, i.e. high motivation, hence the game could be 
proposed as official part supplement to the current syllabus or a more suitable game design could be recommended for 
motivating students. 


1.3 Significance of the Study 


The web-based simulated game for operations management used for this study was for the first time applied in a 
university in Vietnam, as confirmed by the game provider Reponsive Learning Technologies Co. To assess its impact on 
the degree of student-perceived benefits is very necessary because the assessment results of this pilot innovative can be 
used to justify future investments made by the researcher’s university in this educational game and serve as a pilot 
demonstration for other Vietnam’s universities and institutes to adopt or adapt. The focus of this evaluation is 
motivational effect because it is believed that highly-motivated students would be more likely to be engaged in the 
learning process. 

1.4 Relevant Literature 


1.4.1 The Impact of Gaming on Students’ Intrinsic Motivation 

The intrinsic motivation of a person stems from his or her feelings of competency, personal development, and 
self-control experience in his or her work, which arises from within (Ryan & Deci, 2000). Many empirical studies have 
found that games enhance student motivation, including competence, interest or curiosity, and efforts (Burguillo, 2010; 
Cordova & Lepper, 1996; Tuzun, Yilmaz-Sollu, Karakus, Inal, & Kizilkaya, 2008). Vos Meijden Denessen (2011) found 
that the students who learned in the game-based learning environment demonstrated statistically significantly higher 
levels of intrinsic motivation than those who learned in a traditional school environment. 


1.4.2 The Impact of Gaming on Deep Learning Strategy Use and Achievement 

An educational game should aim to have its players adopt deeper learning strategies. During the game, players are 
forced to confront specific problems that they must overcome if they wish to achieve the goal of the game. Students 
accordingly have to develop deep learning strategies rather than just surface learning (Biggs, 1993; Vos Meijden 
Denessen, 2011). Surface-learning students who just memorize information have difficulty solving the hard problems of 
the game; it is necessary to analyze information by “linking it to already known concepts and principles, and leads to 
long-term retention of concepts (i.e. longer memory) so that they can be used for problem-solving in unfamiliar contexts” 
(Vos Meijden Denessen, 2011). Game-based learning can be an effective learning environment for increasing students’ 
levels of deep learning, by critical thinking, problem-solving, decision-making, knowledge transfer and meta-analytic 
skills (Gee, 2003; Kirriemuir & MacFarlane, 2004; Wide man et ah, 2007). 

1.4.3 Problem-Based Learning (PBL) in Web-Based Simulation Games in Operations Management 
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The difference in teaching methodologies between traditional and PBL approaches is highlighted by the statement that 
"the traditional approach centers on lectures given by well-trained teachers who are expert in the field, while the PBL 
approach is based on small groups of students working on tasks under the guidance of a well-trained tutor who is not 
necessarily an expert in the field to be studied” (Keizer, 1995). The PBL approach challenges teams of students to work 
together to solve business problems. The role of the instructor is to facilitate the students (Kanet, 2007). The 
"experiential PBL orientation” and simulation game will ensure a “competition” among teams and serve as a motivation 
for learning. Nadik et al. (2011) used a supply chain game developed by Responsive Learning Technologies Co., in an 
operations management course, and assessed students’ perceptions of their attitudes toward the game. This web-based 
simulation game adopts the PBL approach by integrating requirements for decision making on strategic, tactical, and 
operational levels in supply chain management. Decision topics include demand forecasting, location, lot sizing, reorder 
point, and capacity planning, among others. The findings of a post-game survey revealed that half or more of the 
responding students agreed or strongly agreed that the game increased their skills for forecasting, making decisions in 
inventory planning, working as a team member, and making group decisions. In addition, 72% of students perceived 
that playing the game required more time and effort compared with similar activities in other courses. 

Nevertheless, some studies showed conflicting results about the effects of web-based simulation games. A study 
conducted by IBM revealed that after a game on the internet “most participants did not express interest in websites that 
involved active interaction with the content or other people” and that the “users interacted relatively infrequently with 
the [online] tours, and the less they interacted, the more they reported feeling engaged and entertained by the 
experience.” (Karat et al., 2001). These findings mean that students may be more interested in passive learning than in a 
highly collaborative team game. Since the Littlefield Lab simulation game is a team game on the internet, played for the 
first time at an English-speaking university in Vietnam, it is necessary to evaluate the game’s impact on the students’ 
learning process. The objectives of this study - to assess motivational effects - address that intention. 

2. Method 

2.1 Research Design 

The experiment as conducted quantitatively at the School of Business - International University - Vietnam National 
University in Ho Chi Minh City, Vietnam. The game-learning condition was conducted in summer 2014 and the 
no-game condition was offered in spring 2014. A total of 53 students in game and no-game learning conditions 
responded to an online survey questionnaire. For the game condition, an operations management course taught in 
summer 2014 employed a web-based simulation game designed by Responsive Learning Technologies Co. The students 
were sent an email with a link to an online survey form after they had completed 7 days of playing the game. 
Twenty-nine students completed the survey form. The no-game group, after completing the operations management 
course in spring 2014, did not play the web-based simulation game. Instead, they were assigned problem textbooks and 
in-class exercises. A total of 24 students in the no-game group answered the online survey questionnaire. 

2.2 Instruments 

A 5-point Likert scale was used to measure intrinsic motivation and deep learning strategy use (1 = strongly disagree; 2 
= disagree; 3 = neutral; 4 = agree; 5 = strongly agree). The survey question items were adopted from the motivation 
model of Vos et al. (2011). Intrinsic motivation is composed of three dimensions: competence (4 items), interest in study 
(4 items), and efforts (4 items). Deep-learning strategy use is composed of 4 items. 

2.3 Data Collection and Analysis Process 

After the no-game group finished their operations management course, they were asked to complete the online survey 
form. For the game group, game teams had one week for playing the game and one week for their final report 
preparation. In the second week, team members were sent an email with a link to the online survey. The data analysis 
process included a reliability scale test and analysis of covariance. 

3. Results 

3.1 Descriptive Statistics 

3.1.1 Perceived Judgment on Motivation and Deep-Learning Strategy during the Game 

Table 1 lists perceived intrinsic motivation during the game. On the one hand, less than a half of the students agreed or 
strongly agreed that the game produced competence and interest. On the other hand, some students disagreed or 
strongly disagreed with that fact. Concretely, 24.1-37.9% disagreed or strongly disagreed that the game made them 
competent and 13.8% disagreed that playing the game was quite enjoyable. In terms of efforts, the students’ perceptions 
on the motivational effects of the game were more positive. More than a half of the students (51.7%) agreed or strongly 
agreed that the game made them do their best. Only maximum 13.8% of students did not perceive that they exerted 
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significant effort during the game. In general, more students believed that the game forced them try their best than the 
number of students who disagreed with that fact. In terms of deep-learning strategy use. Table 1 showed that the 
students perceived that playing the game they adopted deeper learning strategy. Concretely, 41.4—55.2% of students 
agreed or strongly agreed that the game made them study more thoroughly; 6.9-37.9% of students disagreed or strongly 
disagreed with that fact. 


Table 1. Students’ perceptions on post-game competence, interest, effort and deep-learning strategy use 


Construct 

Strongly 

disagree 

(%) 

Disagree 

(%) 

Neutral 

(%) 

Agree 

(%) 

Strongly 

agree 

(%) 

Competence (Cronbach’s alpha = 0.902) 

FI. I think I was good at playing the game 

10.3 

27.6 

27.6 

27.6 

6.9 

F2.1 think I did pretty well at playing the game, compared to others 

6.9 

31.0 

24.1 

31.0 

6.9 

F3.1 am satisfied with my performance at playing the game 

0.0 

24.1 

41.4 

34.5 

0.0 

F4.1 am pretty skilled at playing the game 

10.3 

20.7 

37.9 

20.7 

10.3 

Interest (Cronbach’s alpha = 0.864) 

Gl. I think playing the game was quite enjoyable 


13.8 

44.8 

31.0 

10.3 

G2.1 think playing the game was very interesting 

3.4 

10.3 

44.8 

17.2 

24.1 

G3. While I was playing the game I often thought about how much I 

3.4 

10.3 

41.4 

27.6 

17.2 

enjoyed it 

G4.1 think playing the game was boring 

10.3 

48.3 

20.7 

17.2 

3.4 

Effort (Cronbach’s alpha = 0.857) 

HI. I did my best while I was playing the game 

3.4 

10.3 

34.5 

27.6 

24.1 

H2.1 tried very hard to well play the game. 

3.4 

3.4 

51.7 

27.6 

13.8 

H3. It was important to me to play well the game 

3.4 

6.9 

37.9 

41.4 

6.9 

H4.1 put much effort in playing the game. 


13.8 

41.4 

27.6 

17.2 

Deep-learning Strategy Use 
(Cronbach’s alpha = 0.784) 

11.1 found playing this game interesting and want to spend extra time 

13.8 

6.9 

41.4 

27.6 

10.3 

trying to play more games 

12. While playing this game, I asked myself questions to check whether 


6.9 

37.9 

41.4 

13.8 

I understood the problems of the game assignment. 

13.1 want to spend free time playing these kinds of games 

10.3 

27.6 

20.7 

27.6 

13.8 

14.1 thought playing this game was instructive 


13.8 

41.4 

37.9 

6.9 


Note: The purpose of the question item G4 (“I think playing the game was boring.”) is to check the response reliability 
or consistency. When processed in SPSS software, the values of this variable were reversed to be consistent with other 
measures in the group of interest (i.e., 1 becomes 5; 2 becomes 4). The reliability test results showed that all of the 
constructs are reliable (Cronbach’s alpha > 0.7). 

3.1.2 Compared Mean Scores between Game and No-Game Conditions 
Table 2. Comparison of group mean scores for game and no-game conditions 




No-game 

Mean 

Condition 

SD 

Game Condition 
Mean SD 

Competence 

Male 

Female 

3.3889 

3.4667 

0.9109 

0.4364 

3.2500 

2.8529 

1.1180 

0.8797 

Interest 

Male 

Female 

3.0000 

3.3333 

0.6614 

0.4479 

3.2500 

3.5588 

1.0113 

0.7475 

Effort 

Male 

Female 

3.5000 

3.6833 

0.6614 

0.4479 

3.4167 

3.4168 

0.7638 

0.9880 

Deep learning strategy use 

Male 

Female 

3.1389 

2.9833 

0.7512 

0.3467 

3.2083 

3.4706 

0.9405 

0.8380 


Table 2 shows that the average competence score of the game group was lower than that of the no-game group for both 
males and females (3.39 > 3.25; 3.47 > 2.85). However, the average interest score of the game group was higher than 
that of no-game group for both sexes, showing that the learning environment integrated with the game is more 
interesting (3.00 < 3.25; 3.33 < 3.55). The average effort score of the game group was lower than that of the no-game 
group for both sexes (3.50 > 3.41; 3.68 > 3.41). Lastly, we noted that the mean score of the deep-learning strategy use in 
the game group was better that that of the no-game group for both sexes (3.13 < 3.20; 2.98 < 3.47), indicating that the 
game assignment was challenging enough to require the game players to concentrate deeply on their study, as opposed 
to simply completing textbook problems. 


12 




Journal of Education and Training Studies 


Vol. 3, No. 2; 2015 


3.1.3 Correlation between Pre-Game and Post-Game Scales 

The pre-game question items measure the students’ general intrinsic motivation and deep-learning strategy use, based 
on their previous schooling experience; the post-game question items measure the students’ specific intrinsic motivation 
and deep-learning strategy use, based on the game or no-game condition. To test the measurement stability of the 
questionnaire used, we calculated Pearson’s correlations. The correlation calculations showed that post-game 
competence, effort, and deep-learning strategy use were significantly correlated with pre-game competence, effort, and 
deep-learning strategy use (competence r = 0.52, p <0.01; effort r = 0.45, p < 0.01; strategy r = 0.59, p < 0.01). Only the 
“interest” measure did not show a significant correlation before and after the game (r = 0.22, p > 0.01). The correlations 
showed that there was consistency in motivation and strategy use in students. In other words, if a student is motivated in 
his studies in general, he will also be motivated during the game. The next analysis of covariance was to test the 
significance of the impact of the pre-game motivation scales and strategy use on the post-game motivation scales and 
strategy use. 

3.2 Analysis of Covariance 

In this experimental study, we assessed the effects of learning conditions (game and no-game) on student intrinsic 
motivation by comparing the average post-game intrinsic motivation score (competence, interest, effort) and the average 
post-game score of deep-learning strategy use in the two groups, controlling the effects of student pre-game motivation 
and strategy use. Analyses of covariance (ANCOVA) were performed with post-game motivation and strategy use 
scores as dependent variables and the pre-game scores as covariates. 

3.2.1 Impact of Learning Condition on Post-Game Deep-Learning Strategy Use 

Table 3 lists the ANCOVA with the post-game deep-learning strategy score as a dependent variable, the pre-game 
strategy score as a covariate, and the learning game condition as a factor. The ANCOVA results show a product of 
condition and pre-game strategy, F(1.123) = 0.329 > 0.05, meaning that there is no interaction effect between these two 
variables. Table 3 shows that there is a significant difference in mean scores for strategy use between the two condition 
groups (p = 0.027 < 0.05). The higher mean score is in favor of game condition (Table 2). 

Table 3. ANCOVA Result for post-game deep-learning strategy use 


Source 

Type III Sum of Squares 

df 

Mean Square 

F 

Sig. 

Partial Eta Squared 

Corrected Model 

10.601a 

2 

5.300 

18.068 

.000 

.420 

Intercept 

.096 

1 

.096 

.326 

.571 

.006 

Pre-game strategy 

9.713 

1 

9.713 

33.108 

.000 

.398 

Condition 

1.525 

1 

1.525 

5.200 

.027 

.094 

Error 

14.668 

50 

.293 




Total 

562.563 

53 





Corrected Total 

25.269 

52 





a. R Squared = .420 (Adjusted R Squared = .396) 






3.2.2 Impact of Learning Condition on Post-Game Motivation (Competence, Interest, Effort) 

Table 4 shows the ANCOVA with the post-game intrinsic motivation score as a dependent variable, the pre-game 
motivation score as a covariate, and the learning game condition as a factor. The pre-game competence and learning 
condition predict post-game competence (Table 4). However, with pre-game competence as a covariate, the product of 
pre-game competence and learning condition is also significant (F(5.620) = 0.022 < 0.05), meaning that an interaction 
effect exists between these two variables. In “interest,” there is no significant difference in mean scores between the two 
condition groups (p = 0.179 > 0.05) and pre-game interest, the covariate does not influence the post-game interest (p = 
0.070 > 0.05). In “effort,” there is no significant difference in effort scores between the groups (p = 0.569 > 0.05) and 
ANCOVA revealed that the pre-game effort is significantly related to the post-game effort (p = 0.001 < 0.05). 

Table 4. ANCOVA for post-game intrinsic motivation 


post-game competence post-game interest post-game effort 

Type III Sum Type III Sum Type III Sum 


Source 

of Squares 

df 

F 

Sig. 

of Squares 

df 

F 

Sig. 

of Squares 

df 

F 

Sig. 

Corrected 

Model 

11.840a 

2 

12.574 

.000 

2.776a 

2 

2.271 

.114 

5.295a 

2 

6.497 

.003 

Intercept 

1.104 

1 

2.346 

.132 

9.466 

1 

15.488 

.000 

7.184 

1 

17.629 

.000 

condition 

2.152 

1 

4.571 

.037 

1.137 

1 

1.861 

.179 

.134 

1 

.329 

.569 

pre-game 

9.425 

1 

20.017 

.000 

2.091 

1 

3.421 

.070 

5.123 

1 

12.571 

.001 

Error 

23.542 

50 



30.557 

50 



20.375 

50 



Total 

579.063 

53 



622.222 

53 



694.313 

53 



Corrected 

35.382 

52 



33.333 

52 



25.670 

52 




Total 
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a. R Squared = .335 (Adjusted R Squared = .308) 

4. Discussion 


a. R Squared = .083 (Adjusted R a. R Squared = .206 (Adjusted R 
Squared = .047) Squared = . 175) 


We used a quasi-experimental approach to examine the influence of a web-based simulation game, played in an 
operations management course, on student intrinsic motivation and use of deep-learning strategy. An ANCOVA 
technique was used to test the significance of this impact and the interaction effect between pre-game motivation and 
strategy use on learning conditions (game or no-game). 

4.1 Deep-Learning Strategy Use 

We found that playing the game forced the game group to study more thoroughly than the no-game group; this 
difference was significant. Interaction effect testing revealed that pre-game strategy use, as a covariate, is not 
significantly related to post-game strategy use. This fact is a positive point from the game because it is believed that 
deep learning contributes to longer retention of concepts. 

4.2 Intrinsic Motivation (Competence, Interest, Effort) 

There was no significant difference between the two condition groups in mean scores for interest and effort. The results 
challenged the assumed motivational advantages thought to result from the game method versus the traditional no-game 
method. More interestingly, the mean score in competence for the no-game group was better than that of the game 
group; this difference was significant. 

These results surprisingly do not support the idea that the motivational effects of a game will have a positive impact on 
students. The students did not exert a significant effort playing the game, which may be due to several possible reasons. 
If students are not interested, they are not engaged and they will not exert an effort. It is also possible that the 
game/website design and game grading were not demanding enough to force the students to try their best. 
University-level students may furthermore be less interested in playing games than primary-level students. Additionally, 
students might not have felt competent at playing the game maybe because it could have been inconvenient or harder 
than just opening a textbook and doing written exercises. The learning style of each individual student may also 
contribute to our findings. An IBM evaluation found that students tend to be passive leisure game players, rather than 
frequently interacting with other team members (Karat et al., 2001). The game in this study is highly demanding in 
terms of team dynamics, which may explain why the students were not very interested or engaged. 

There are also a number of further reasons that may explain our findings. First, a majority of the participants had not 
taken any quantitative courses, as confirmed in class. Since this simulation requires quantitative decision-making 
techniques such as forecasting and capital budgeting, students without this background may find the game particularly 
challenging. Second, most of the students were learning for the first time how to run a business based on maximizing 
profits; the majority of students were sophomores. If the game was run a second time, the students would likely perform 
much better. Third, since the game lasted only one week and the students played the game on their own time, the 
researcher could not monitor how often each of the students accessed the website to play the game. Finally, since the 
game represented teamwork, it is impossible to avoid social loafing, a phenomenon in which some students take 
advantage of the competence and efforts of other more enthusiastic students. This phenomenon may have led to the low 
levels of perceived competence and effort. Worse, if the keen and competent students recognized the free-riders, the 
more advanced students might have felt the lack of equity when all team members equally enjoyed the same team score. 

5. Conclusion 

The analysis results suggested that designing a web-based simulation for enriching operations management education 
should consider some issues such as game website attractiveness, access monitoring system, individual performance 
assessment. Assessment of team performance needs to be supplemented with member performance evaluation to ensure 
fairness in marking. The use of this simulation needs to be officially integrated in the operations management course 
syllabus for consistence over academic semesters. In addition, students registering operations management course 
should have prior knowledge on quantitative analysis so that they can apply quantitative techniques for making 
decisions in this simulation. 


One limitation of this study is it relatively small number of participants (53). Qualitative research, with in-depth 
interview or focus group discussion, should be included in future studies to examine critical issues during the game. 
Games are believed to bring about motivational effects but this study did not provide evidence for this fact. For example, 
how much do good teamwork and dynamics influence team effectiveness? How does social loafing impact teamwork? 
Also, technological and psychological barriers need to be identified. We recommend that future research focus on the 
following issues. First, a Technology Acceptance Model should be used to explore factors affecting students’ behavior 
to use the game website. Some factors, such as ease of use and perceived usefulness, are believed to be related to users’ 
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behavior to accept a technology product (or website in this case). Second, education researchers should investigate how 

passive learners are enjoying a game that requires a high level of teamwork. This research direction is important for 

educational contexts in Asia, where students traditionally experience passive learning approaches. 
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